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SYSTEM AND METHOD FOR EXTRACTING A FACE FROM A CAMERA PICTURE 
FOR REPRESENTATION IN AN ELECTRONIC SYSTEM 

The present invention relates generally to computer vision technology for 
5 recognizing images, and more particularly, to computer vision technology that extracts an 
image, such as a person's face, from a picture, for representation in an electronic system. 

In present and future electronic systems, people are/will be able to meet 
each other online, in a so-called "virtual space" and, in order to identify themselves in this 
virtual space, they need to be represented by a personal icon. For example, a virtual space 

10 may be realized by any P2P system (e.g., global networking). 

In another example, at the entrance of an office building a display could 
show a map of all rooms including pictures, i.e., iconic representations, of the persons who 
are currently in each room. The question arises how these kind of 'person icons' may be 
easily captured, and kept up-to-date. 

15 Face recognition systems may be employed for determining faces in an 

image. A representative face recognition system is described in U.S. Patent No. 6,108,437 
which is directed to a feature extraction system that processes an image for determining 
whether a face is present in an image and for determining its position in the image. One 
application of a face recognition system is described in U.S. Patent No. 5,781,650 which is 

20 directed to a system for determine facial features from an extracted image in order to 
classify a person's age from his/her image. U.S. Patent No. 5,987,154 entitled "Method 
and Means for Detecting People in Image Sequences" is directed to a system for locating a 
person in a sequence of video images to facilitate visual communication. International 
Patent Publication WO-2003 10728 is another application of a face recognition system that 

25 is targeted at privacy safeguarding. Its aim is to hide identities of people/objects by first 
detecting the presence of a person's personal identifying characteristics present in an 
image, and then automatically removing or replacing those characteristics. 

Most face recognition systems use computer vision for recognizing identity 
of people and objects and these systems do not output (parts of) the image itself. The 

30 reference WO-2003 1 0728 is the only exception to this, but here the goal is to hide the 
identity of a person. 

The prior art computer vision systems are thus deficient in that none of 
these systems is used to capture and process an image of a person in order to use it to 
represent the person in another system, e.g., a PCC or, like system requiring interfacing of 
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humans and machines. Thus, it would be highly desirable to provide a computer vision 
system that aims to present an image of a face as representation of a person's identity in an 
organization or system. 

It is an object of the present invention to provide a computer vision system 
5 and methodology that presents an image of a face as representation of a person's identity in 
an organization or system. In accordance with this object, the computer vision system and 
method addresses the timing/selection of a desired image. For instance, an image may be 
selected in which the subject person is gazing at the camera, or in which the person is 
smiling, etc. 

10 Accordingly, a computer vision system and methodology is provided that 

implements includes face recognition for capturing a picture of the person that needs to be 
represented in another electronic system that uses or presents images, for example, and to 
extract or 4 cut out' the face from that picture. The system and method is further able to 
take the snapshot on the moment that the person is gazing straight towards the camera or 

15 even at the moment that a person's facial appearance is in a desired state, e.g., is smiling, 
and render the extracted image suitable for representation in another electronic system. 
Thus, the inventive system addresses the timing/selection of a desired image to be 
extracted. 

Also provided are a computer program product for carrying out the methods 
20 of the present invention and a program storage device for the storage of the computer 

program product therein. The computer program product can be embodied in a computer- 
readable medium or contained in the multi-media stream. 

These and other features, aspects, and advantages of the apparatus and 
methods of the present invention will become better understood with regard to the 
25 following description, appended claims, and accompanying drawings where: 

Figure 1 illustrates a schematic view of an embodiment of an apparatus for 
detecting and extracting an image from a video camera output (digital stream); 

Figure 2 illustrates one application for using an extracted portrait of a 
person for identifying that person as an icon in a personal content container device 
30 connectable via a WiFi network. 

The most natural way of representing a person is by a picture of the person's 
face. However, it is hard for a person to position himself in front of a camera in such a 
way that the picture has the appropriate properties of a portrait photo. Referring now to 
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Figure 1, there is illustrated a schematic of an embodiment of an apparatus 10 for detecting 
a person in an image. 

According to the invention, a first step is processing of an image to check 
for the presence of a face (using facial features). For example, as shown in Figure 1, a 
5 camera device 15 adapted to generate still image or full-motion video images generates, 
respectively, a first real-world image 12 or series of images (an image sequence), e.g., of a 
person or group of people, for example. This image may be digitized and converted to a 
digital stream 18 for input into a image recognition processor or like device 20, for 
example, which may apply facial recognition to capture a snapshot 22 of the target 
10 person's face from the first (larger) image before extracting it. As known to skilled 

artisans in face recognition and computer vision generally, the digital stream of the image 
or image sequence may be parsed for detecting in the datastream various sub-strings which 
may comply with certain substring characteristics, viz., the characteristics of a human face. 
Then, computer vision techniques are utilized to "cut out" the face from a possible larger 
15 picture. It is understood that the camera device 15 and image processor 20 may be separate 
devices or embodied as a single stand-alone unit. 

In one embodiment, a time of the taking and receipt of the still image/video 
image may be programmed, and determinable by computer vision, such as at the moment 
when a person is smiling or when gazing towards the camera device. That is, the image 

20 sequence may extract the snapshot of the target person's face 22a when the face in the 
image is smiling or fully faces the camera. Then, the extracted image 22a is stored in a 
memory storage system or database 25, comprising non- volatile (or volatile) memory 
storage media. The extracted image may be stored together with contextual data/meta data 
such as time, place, camera, and if known that person's "personID". Finally, according to 

25 the invention, the stored image 22a and associated contextual/meta data may be 

subsequently (or immediately) imported to or retrieved from the same system or second 
distinct system 30 executing any application requiring visual identification of the person. 
In one example, for instance, as depicted in Figure 2, the stored image 22a may be 
displayed as an icon 22b on a screen display 45 of an electronic system 30, e.g., a mobile 

30 telephone 50. Otherwise, the extracted image may be printed on paper in a later processing 
stage. 

According to the invention, the above-described principle may be applied in 
various camera/computer vision systems: 
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1) a smart security camera that is adapted to create a logging list of 'portrait 
photos' of people entering or leaving a particular building; 

2) a portable "infotainmenf ' (content used for information or entertainment) 
players such as the Portable Content Container (PCC) may support use of user icons in an 

5 ad-hoc sharing situation; 

3) a PC or TV camera could be adapted to use the extracted image to create 
user icons that could be used to represent a particular person; e.g., representation in virtual 
community spaces (with advantageously, the feature that the extracted photo is up-to-date); 

4) a PC, TV or bathroom mirror display system may be adapted to include 
10 an integrated camera that makes a daily snapshot of the person's face. This allows the 

person to playback a sequence of images of his/her face, to see how his/her face has 
changed over time; 

5) a camera-equipped mobile phone may be used to quickly make a 
snapshot and to use this picture as a representation of the person's telephone number (caller 

15 ID, address book etc). The picture would pop up if this person calls the user's phone, for 
instance; and 

6) a camera at the entrance of a building or a reception may make snapshots 
of people entering so that these could be used to instantly create a visitor's badge using a 
camera and computer vision at the entrance/reception of a company building. For 

20 example, a 'face scanning system' may be employed to continuously scan faces of people 
entering to see if they are new, or make a snapshot on explicit request. This is especially 
applicable for an organization requiring user identification badges. That is, this system is 
useful if badges with photos can be printed instantly and especially when future display 
technologies become cheap and enable a visitors' badge to be electronically updated with a 

25 recent picture of the visitor or employee. This means that an employee badge could 
(theoretically) be updated every day. 

An additional feature may include the employment of the system 
programmed to frequently capture portrait pictures of a person so that a face recognition 
system may be kept up-to-date and to gradually adapt the recognition pattern to be able to 

30 recognize a person over a long time. For instance, the invention may facilitate and render 
possible the maintenance of an electronic photo frame of a target user that may be 
frequently updated (e.g., daily) with a recent picture of the target user. 
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Those skilled in the art will appreciate that the present invention may be 
usefully employed in other systems where a facial image may obtained from a first image 
and subsequently output in a useful form for representation in another system as a separate 
smaller image. 

5 While there has been shown and described what is considered to be 

illustrative embodiments of the invention, it will, of course, be understood that various 
modifications and changes in form or detail could readily be made without departing from 
the spirit of the invention. It is therefore intended that the invention be not limited to the 
exact forms described and illustrated, but should be constructed to cover all modifications 
10 that may fall within the scope of the appended claims. 



